Papers with neural commonsense model
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models (2022.naacl-main)
Copied to clipboard
Peter West, Chandra Bhagavatula, Jack Hessel, Jena Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, Yejin Choi
| Challenge: | Prior studies suggested pre-trained language models possess limited understanding of commonsense knowledge despite otherwise stellar performance on leaderboards. |
| Approach: | They propose a framework that uses larger models to teach smaller models by distilling knowledge symbolically as text in addition to the neural model. |
| Outcome: | The proposed framework is based on a general language model teacher's commonsense knowledge graphs and a neural commonsensing model surpassing the teacher model's in all three criteria. |